Record which routes can page by keyset at schema 5, and hold page_token to them in R-PAGE - #1574

…#1642)

Refs #1595
Refs #1604
Refs #1605
Refs #1606
Refs #1607
Refs #1608
Refs #1609
Refs #1610
Refs #1611
Refs #1612
Refs #1613
Refs #1614
Refs #1615
Refs #1616
Refs #1617
Refs #1618
Refs #1209
Refs #1619
Refs #1620
Refs #1621
Refs #1622
Refs #1623
Refs #1624
Refs #1625
Refs #1626
Refs #1627
Refs #1628
Refs #1629
Refs #1630
Refs #1631
Refs #1632
Refs #1633
Refs #1634

This is the seventh layer of the auditor stack. The sixth added R-ROUTE
and gated on stale declarations only, which left 730 routes of the
19.5.0 record that no action sends and nothing answered. This layer
answers every one of them and turns on the full gate.

## What the check detects

`-scope=routes` now fails on a finding of any class (a route nothing
answers, an unknown skip reason, an internal route left in, a
contradicted decision), as it already did on a declaration that no
longer describes the tree. The gate reads those classes from
`findingClasses`, so a class added there is gated without a second list
to keep in step. A re-pin that brings a route, or an action that stops
sending one, fails the gate until the change that causes it decides the
route, and every family row pins how many routes it answers.

One declaration problem is new: a `tracked` row over a route whose
client-go method `acceptedMissingMethods` decides `COVERED_GENERIC` or
`INTENTIONAL_SKIP_*`. The row says an action is still to implement and
the decision that the route needs none, so one of the two no longer
describes the tree.

## How the routes are answered

I applied the decisions of 2026-10-27. `covered-by-graphql` is a REST
route whose operation an action does over GraphQL, the action checked in
the catalog and in the request record: 49 routes. They are the `/-/`
epic routes and the child epic links, the five work item operations and
a child's attachment and detachment behind `work_item_rest_api`, a
group's work item's discussions, notes and children (13 routes the
`group.epic_*` actions cover, since their GraphQL reads the group's work
item by its iid whatever its type), a vulnerability's confirmation,
dismissal, resolution, revert and read, and a pipeline's findings. A
project's and a namespace's work item discussions, notes and children
stay tracked, and so does the resolution of a discussion, which no
action does.

`client-protocol` is the protocol a package manager's or MLflow's own
client speaks, 150 routes, except a download, which is tracked. The line
is what the route serves, not how GitLab words its summary: what a
package or a model version holds (one of its files, its archive, its
source or a file inside it, such as a Go module's `go.mod`) is a
download, while an index or a metadata document a client reads to find
and resolve packages, and the URL or the location of the content, are
the protocol's. So Debian's `Release` and `Packages` files, Helm's
`index.yaml`, RPM's `repodata` and RubyGems' spec indexes and gemspecs
are `client-protocol`. The witness holds every `client-protocol` route
whose summary says download to a list that names what each serves
instead, and every entry of that list and every protocol root to a route
it answers.

`tracked` is everything else, 534 routes, under 32 issues: one for each
of thirty catalog domains and one for each of two domains the catalog
does not hold yet, the virtual registries and GitLab Duo. The fifteen
`public_endpoint` routes and every administrator, GitLab.com and tier
route are among them, and a route's reason says it is public exactly
when its skip reason is `public_endpoint` (`POST /glql` is not; `GET
/glql/schema` is).

Seventeen of the tracked routes were settled by
`acceptedMissingMethods`: six `COVERED_GENERIC` (a group's and a
project's direct members, a group's subgroups, the delete of an LDAP
link by its `cn`, a merge request's changes and the `PUT` of the error
tracking settings) and eleven `INTENTIONAL_SKIP_*` (nine downloads, the
streamed repository archive and the deprecated write of a merge
request's approvals). A route still to implement is no method R-ACTION
may settle, so those entries leave the table and R-ACTION lists the
seventeen methods as its backlog, 18 missing methods with
`TerraformStates.List`, which was there already; `Orbit.GetDsl` was the
nineteenth until the Orbit migration, and is decided
`SUPERSEDED_UPSTREAM` now. The creation of a vulnerability keeps its
`COVERED_GRAPHQL` service decision, which does not carry because no
GraphQL action creates one. The two runner token routes and the
integrations list carry no decision at all: they are tracked rows whose
reason names `runner_token_auth` or the `/services/` twin.

The table moved to `family_table.go`, written with five constructors
(`tracked`, `download`, `publicEndpoint`, `protocol`, `overGraphQL`). It
has 331 rows, more than the design estimated, because no two rows may
answer one route: downloads sit inside the protocol trees, so each
protocol is split around its downloads, and a group's work item routes
are split from a project's and a namespace's.

The Orbit rows are as the Orbit migration (#1585) left Orbit, all under
#1209, the issue that already tracked the Orbit routes no action
reaches. The five routes the `orbit.*` actions sent until GitLab.com
removed them (`GET /orbit/status`, `/orbit/schema`, `/orbit/tools`,
`/orbit/schema/dsl` and `/orbit/graph_status`) inherit
`SUPERSEDED_UPSTREAM` from the layer below, and `POST /orbit/mcp`, which
the actions send in their place, is reached, so the `/orbit/mcp` row
tracks the endpoint's SSE listener alone, `GET /orbit/mcp`. `GET
/orbit/agent/commands`, which `orbit.dsl` read until GitLab.com removed
it too and which no client-go method sends, shares a row with the
invocation of a command; the query templates, the named templates, the
response format guidance and the skills stay tracked, the reasons of the
two GitLab.com removed in favour of GraphQL saying so.

## Witness tests, red then green

On the parent as it stood before the stack was rebased onto #1585
(9a1b315fb), with this layer's tests, a stand-in for the four
identifiers they need (two issue constants, `publicTail` and
`Summary.counters`) and without its table or gate:

```text
$ go test -count=1 -run "TestSummary_Clean|TestJudgeAll_AFamilyRow|TestRun_Repository" ./cmd/audit_1to1/internal/routes/
--- FAIL: TestSummary_Clean_PassesOnlyWithoutAFindingOrAStaleDeclaration (0.00s)
    --- FAIL: TestSummary_Clean_PassesOnlyWithoutAFindingOrAStaleDeclaration/an_unanswered_route (0.00s)
        analyze_test.go:301: clean({... Unanswered:1 ...}) = true, want false
    (and the unknown skip reason, the two internal classes and the contradicted decision)
--- FAIL: TestSummary_Clean_FailsOnEveryFindingClassAndOnNoOther (0.00s)
    --- FAIL: TestSummary_Clean_FailsOnEveryFindingClassAndOnNoOther/unanswered (0.00s)
        analyze_test.go:318: one route of class unanswered: clean() = true, want false
    (and the other four finding classes)
--- FAIL: TestJudgeAll_AFamilyRow_AnswersItsRoutesUntilTheyChange (0.02s)
    --- FAIL: TestJudgeAll_AFamilyRow_AnswersItsRoutesUntilTheyChange/a_tracked_row_over_a_route_a_COVERED_GENERIC_decision_settles (0.00s)
        analyze_test.go:575: stale = [], want ["family GET /gizmos tracks GET /gizmos, and Gizmos.ListGizmos is decided COVERED_GENERIC, which says the route needs no action of its own: an action still to implement is no method acceptedMissingMethods settles"]
    (and the INTENTIONAL_SKIP_BINARY case, and the cases whose gate the parent passes with findings left)
--- FAIL: TestRun_Repository_DeclaresTheRoutesADecisionDoesNotCarryOnto (0.22s)
        analyze_test.go:960: GET /groups/:id/members = {Route: Class: ...}, want it tracked by a family row whose reason names "", with the decisions "Groups.ListGroupMembers unadjudicated"
    (and the other 21 routes of the witness)
--- FAIL: TestRun_Repository_HoldsTheDuoWorkflowRoutesToTheTokenThatCallsThem (0.16s)
    analyze_test.go:1008: composite identity = [...], 0 a user's token calls tracked by the Duo row and [... unanswered ...] otherwise, want [...], 14 and none
--- FAIL: TestRun_Repository_AdjudicatesEveryRouteNoActionReaches (0.16s)
    analyze_test.go:1089: declared map[tracked:3] under 2 issues; want map[client-protocol:150 covered-by-graphql:49 tracked:535] under one issue for each of 30 catalog domains and of the virtual registries and GitLab Duo, which it does not hold yet
    analyze_test.go:1092: 0 public_endpoint routes tracked, want 15; ...
    analyze_test.go:1095: ...; roots and listed downloads no client-protocol route answers: ["/group/:id/-/packages/" "/groups/:id/-/packages/" "/packages/" ...]
--- FAIL: TestRun_Repository_HoldsTheWorkItemRoutesToTheActionsThatDoTheirOperation (0.14s)
    analyze_test.go:1174: work item routes by scope and category = map[], want map[groups:map[covered-by-graphql:20 tracked:10] namespaces:map[covered-by-graphql:7 tracked:23] projects:map[covered-by-graphql:7 tracked:23]]; ...
FAIL
```

Since the rebase the witness of the decisions that do not carry holds 21
routes, not 22, and the count it wants is 534 tracked: `GET
/orbit/schema/dsl` is inherited now, with the other four Orbit routes
GitLab.com removed.

With this layer's gate check and its seventeen `acceptedMissingMethods`
entries still in place, `TestRun_Repository_GateIsGreen` names the
contradiction on the real tree, six `is decided COVERED_GENERIC`, nine
`is decided INTENTIONAL_SKIP_BINARY` and two `is decided
INTENTIONAL_SKIP_OTHER`:

```text
    analyze_test.go:837: the gate fails: 0 routes no action reaches are left findings, among them [], ...; and the stale declarations ["family * /groups/*/uploads/** tracks GET /groups/:id/uploads/:secret/:filename, and GroupMarkdownUploads.DownloadGroupMarkdownUploadBySecretAndFilename is decided INTENTIONAL_SKIP_BINARY, which says the route needs no action of its own: ..." ...]
```

Here:

```text
--- PASS: TestSummary_Clean_PassesOnlyWithoutAFindingOrAStaleDeclaration (0.00s)
--- PASS: TestSummary_Clean_FailsOnEveryFindingClassAndOnNoOther (0.00s)
--- PASS: TestJudgeAll_AFamilyRow_AnswersItsRoutesUntilTheyChange (0.02s)
--- PASS: TestRun_Repository_GateIsGreen (0.88s)
--- PASS: TestRun_Repository_HoldsTheRoutesClientGoLaterModels (0.20s)
--- PASS: TestRun_Repository_DeclaresTheRoutesADecisionDoesNotCarryOnto (0.22s)
--- PASS: TestRun_Repository_HoldsTheDuoWorkflowRoutesToTheTokenThatCallsThem (0.27s)
--- PASS: TestRun_Repository_AdjudicatesEveryRouteNoActionReaches (0.26s)
--- PASS: TestRun_Repository_HoldsTheWorkItemRoutesToTheActionsThatDoTheirOperation (0.25s)
--- PASS: TestDeclares_ATrackedRowOverARouteADecisionSettles_IsAProblem (0.00s)
```

The witness is sensitive to the version on purpose: the client-go bump
and every re-pin move its counts, and that change is the evidence the
triage was run again.

## On main

2137 routes: 1007 reached, 20 outside the API, 186 skipped, 94 by a
slug, 49 aliases, 48 inherited and 733 declared by 331 rows, with no
finding and no stale declaration. Each tracked route is in the text of
its domain's issue with its row, its summary and the client-go methods
that send it, if any.

## Checks

`go test ./... -count=1` with the snapshot parity deferred, and the
`audit_1to1` suites on the real tree without it. 100% statements and
every condition both ways on `routes` (gobco 242 of 242) and on
`actions` (22 of 22); gremlins Lived 0 and Not covered 0 on `routes`
(181 killed) and on `actions` (16 killed). `golangci-lint` is clean on
`cmd/audit_1to1/...`, and the `make analyze` steps the layer touches
pass: the routes, params and paths gates, the sdk gate, the subtest,
goroutine, unread-constant, closed-server, godoc and test-name gates,
the documented tool names and `golangci-lint fmt`, as do markdownlint,
the table formatter, the doc link check and the em dash check.
`gen_testing_docs --check` is regenerated at the top of the stack, as
below this layer. The two pieces of work this layer kept clear of have
merged since, the keyset half of the live record (#1574) and merge
request diff pagination (#1584), and so has the Orbit migration (#1585),
which stopped sending five Orbit routes this layer counted reached (`GET
/orbit/agent/commands`, `/orbit/graph_status`, `/orbit/schema`,
`/orbit/status` and `/orbit/tools`) and started sending `POST
/orbit/mcp`, which the `* /orbit/mcp` row tracked. The stack is rebased
onto main after all three, the Orbit rows are redone as above, and every
figure in this description is measured there except the red run, which
is quoted from the parent before that rebase.

---------

Signed-off-by: José M. Requena Plens <mail@jmrp.io>

… toolutil declares (#1643)

Refs #1595
Refs #1603

This is layer A8 of the auditor stack (H7). The type grain of
`-scope=paths` now holds every compared type one level down in both
directions, and the R-OUTPUT field diff reads the converters
`internal/toolutil` declares. Nothing new gates: what reaches the gate
is still a declaration that answers nothing, now at either level.

## What it detects

The nested comparison compared 44 nested types, all of them under the 68
compared types no struct names: `publishedTypesIn` filled `Nested` only
for those, and a payload is named by its envelope, so the other 184
compared types, the merge request a list wraps among them, were never
held one level down. That is how `head_pipeline.name` stayed published
on every merge request although `API::Entities::Ci::Pipeline` sends no
name. The walk now records what every type nests: 294 nested types
compared. The package grain keeps reading one level down from a
top-level type, since it joins on names alone and a name two levels down
silenced nine of its findings.

The nested level now asks the reverse too
(`shapes.typed.nested_unsurfaced`): a key the entity under a property
sends that the nested type does not publish, read on the entity the
record's `using` names, key by key as the top level is, since two routes
of a type can render one property with two entities, with `sdk_models`
from the client-go struct under the same key (`OutputPairing.SDKNested`,
and the same on projections). Both declaration tables gain `Under`, and
a declaration answers a nested finding only when it names the property.
One nested type under one property of two types of a package is reported
once for each, so both finding lists now close their order on the
client-go struct and the endpoints each finding was judged against: four
runs of `-scope=paths` over this tree write the same bytes.

The R-OUTPUT diff reads `internal/toolutil` and reports it as a
`toolutil` entry. `toolutil.NewPipelineOutput` fills `head_pipeline` and
no domain package declares a converter of it, so a key client-go adds to
`gl.Pipeline` reached no finding. `CollectPairs` and
`CollectOutputPairings` are unchanged, so R-ENUM and the type grain do
not move; passing the shared pairs to R-ENUM is left unmeasured here.

## Witness tests


`TestTypedShapeCheck_Repository_HoldsTheHeadPipelineToThePipelineEntity`
(paths, real tree), red on the parent a81f7cd28, where its first
commit's version, the `name` half, copied in on its own fails with `no
nested finding for internal/tools/mergerequests head_pipeline.name, want
toolutil.PipelineOutput's name reported unpublished`, and the same for
`deploymentmergerequests`. Green here, with `head_pipeline.archived`
reported on both, read on `API::Entities::Ci::Pipeline`, sent always and
`sdk_models` false. The parent's report has no `nested_unsurfaced` and
names no `archived` for a merge request. The bump turns `sdk_models`
true, since `gl.Pipeline` models `archived` from v3.36.0.

`TestBuildReport_Repository_ReportsTheConvertersOfTheSharedShapes`
(structs, real tree), copied on its own into the parent, fails there
with `package "toolutil" not found in report` (the parent's report has
153 packages). Green here: a `toolutil` entry exists, and
`PipelineOutput` against `gl.Pipeline` carries the same keys in v3.30.0,
three of them with a type of its own (`detailed_status`, `source` and
`user`), which the bump turns into a missing `archived`.
`TestBuildReport_Repository_PairsTheConvertersTheSharedShapesDeclare`
checks that pair comes from the shared package's own converter, and
`TestCollectOutputPairings_Repository_ReadsTheStructsAMergeRequestNests`
pins `name` and no `archived` under `gl.MergeRequest`'s `head_pipeline`.

Fixtures cover each rule: a list's payload has its part judged; the
reverse is reported under its property and read on the entity that sends
the key, also where two routes, or two routes merged into one shape,
render the property with two entities; `sdk_models` reads the same
property only; a projection answers it from its struct; a declaration
with `Under` answers its finding, goes stale once the key is published,
and is held to its evidence like a top-level one; the summary counts the
nested level apart from the top; two types nesting one type report their
findings in one order; the package grain does not credit what an inner
type nests; and a stand-in `toolutil` converter of `gl.Part` reports
`archived` missing once a stand-in SDK adds it to `Part`, while the SDK
as it stands leaves the pair without a gap.

## Findings on main and how each is answered

`head_pipeline.name` and `head_pipeline.archived` on `mergerequests` and
`deploymentmergerequests` are the witness cases; `archived` is published
in the client-go bump stack, which models it.

The nested level reports 111 published keys GitLab does not send under
their property and 1085 keys it sends that the nested type does not
publish, 642 of them not modeled by client-go either. None gates, and
none is declared here: the design leaves the nested findings beyond
`archived` to B4 or to an issue, so they are listed by nested type in
the follow-up. `summary.typed_undeclared_fields` reads 111, all at the
nested level.

R-OUTPUT gains 42 output pairs with 29 missing and 20 extra fields and
17 type mismatches under `toolutil`, reported like the domain packages'
387 missing and 85 extra fields and 593 type mismatches, and one more
package with gaps (60, from 59). No declaration table goes stale.

## Checks

All green: `go test ./... -count=1` with
`GITLAB_MCP_TEST_SNAPSHOT_PARITY=deferred`, the audit_1to1 tests on each
commit, golangci-lint, markdownlint, the doc link check,
`format_md_tables --check`, `make check-test-file-names`, `make
check-em-dash` and `make check-pr-description`, and `make analyze` steps
2, 4, 7, 8, 14, 32 and 33, plus `-scope=sdk`, `-scope=structs` and
`-scope=enums`. Four runs of `-scope=paths` over the head write the same
bytes.

Statement coverage 100% in `paths` and `structs`; conditions 1060/1060
and 728/728 (gobco). Mutants: `structs` 279 killed, 0 lived, 0 not
covered (the 2 timed out are in `cross_call.go`, which this layer does
not touch); `paths`, limited to the files this layer changes, 286
killed, 0 lived, and 420 not covered, every one an `ARITHMETIC_BASE` `+`
in a package-level const or var initializer of `sent_declarations.go`
(331), `shape_declarations.go` (85) or `published.go` (4), which Go's
coverage does not instrument, and none on a changed line.

## Overlaps

#1584, which carried #1517 into main, added two entries to
`declaredUnsurfaced` for the merge request the two note packages read to
check an inline position. This layer adds the `Under` field and changes
no entry, so an entry without `Under` keeps its meaning, and the rebase
onto main was mechanical: the conflicts were the import block of a test
file and a comment of `shapes.go`, both of which #1574 had changed. No
`apilive` or `gen_api_live` file is touched.

---------

Signed-off-by: José M. Requena Plens <mail@jmrp.io>